Human interface method and apparatus

ABSTRACT

A method and apparatus for measuring one or more physical conditions of a computer operator and for automatically inputting signals corresponding to the physical conditions into a computer ( 12 ) for control and monitoring purposes.

CROSS REFERENCE TO RELATED APPLICATION

This application is a Continuation in Part of U.S. Pat. No. 09/567,176titled “Human Interface Method and Apparatus” whose inventor is KenTamada, which was filed on May 8, 2000, now abandoned.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to computer interface devices and methods.More particularly, the invention relates to a method and apparatus formeasuring one or more physical conditions of a person and inputtingsignals corresponding to the physical conditions into a computer forcontrol and monitoring purposes and for permitting computer users tointeract with other remote computer users via a communications network.

2. Description of the Prior Art

Many computer programs are designed for use by computer operators havingdifferent levels of experience or knowledge. For example, many computergames have different “levels” or speeds that accommodate players ofdifferent skill levels. Similarly, many computer application programshave various features ranging from simple to complex that may be used bydifferent persons depending upon their skill levels.

Computer operators can currently only interact with a computer byinputting data or information into the computer via a keyboard, computermouse, or other manually operated input device. These types of computerinput devices cannot determine information about a computer user andthen automatically input the information but instead rely upon theoperator to manually enter all information. Thus, computer operatorswishing to interact with a computer program based on their skill orknowledge level must currently manually input such information into thecomputer. For example, a computer user having a high skill level for aparticular computer game currently must manually enter informationrelating to their skill level to cause the computer program to operateat a higher level or speed.

Another problem with conventional computer input devices is that theycannot automatically enter sensory information into a computer relatingto a physical condition of an operator. This is a problem because manycomputer users work in stressful conditions in which their currentphysical condition may be an important factor. Also, current computerinput devices do not permit computer users to interact with other remotecomputer users without manually entering information.

OBJECTS AND SUMMARY OF THE INVENTION

The present invention solves the above-described problems and provides adistinct advance in the art of computer input devices. Moreparticularly, the present invention provides a method and apparatus formeasuring one or more physical conditions of a person with a sensor andautomatically inputting sensor signals corresponding to the physicalconditions into a computer for control and monitoring purposes and forpermitting computer users to interact with other remote computer usersvia a communications network.

One embodiment of the method of the present invention includes the stepsof measuring with a sensor worn or touched by the user a physicalcondition of the user when the user is in a rest state and generatingcorresponding rest state sensor signals; inputting the rest state sensorsignals into the computer; running a computer program on the computerthat requires interaction by the user; measuring with the sensor thephysical condition of the user when the user is interacting with thecomputer program and generating corresponding active state sensorsignals; inputting the active state sensor signals into the computer;comparing the rest state sensor signals to the active state sensorsignals; and changing some aspect of the computer program when theactive state signals differ from the rest state signals by apredetermined amount.

These and other important aspects of the present invention are describedmore fully in the detailed description below.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

A preferred embodiment of the present invention is described in detailbelow with reference to the attached drawing figures, wherein:

FIG. 1 is a schematic diagram showing a human interface deviceconstructed in accordance with a preferred embodiment of the presentinvention shown coupled with a personal computer.

FIG. 2 is a schematic diagram of several human interfaces and personalcomputers shown coupled with a host computer via a communicationsnetwork.

FIG. 3 is a block diagram illustrating several core components of acomputer program that may be used with the human interface of thepresent invention.

FIG. 4 is a flow diagram depicting certain steps of the computerprogram.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Turning now to the drawing figures, and particularly FIG. 1, the presentinvention broadly includes a human interface device 10 and a computerprogram that may be run on a computer 12 such as the one illustrated.The human interface device 10 measures one or more physical conditionsof a computer operator and automatically inputs signals corresponding tothe physical conditions into the computer 12. The computer program thenanalyzes the signals to control certain aspects of other computerprograms run by the computer 12 or for monitoring purposes as describedin more detail below. The signals may also be used to permit computerusers to interact with other remote computer users via a communicationsnetwork.

The human interface device 10 includes one or more sensors 14, a carrier16 that permits the sensors to be worn by an operator of the computer12, and a computer interface 18 for coupling the sensors with thecomputer 12. The sensors 14 may be any commercially available sensorsthat measure physical conditions such as blood pressure, pulse rate,body temperature, static capacity, skin conductivity, brain waveactivity, NO₂ or NO₃ blood density, static electricity, or bloodcharacteristics. The sensors 14 each preferably include a signalamplifier and an analog-to-digital converter for producing digitalsensor signals corresponding to the measured physical conditions.

The sensors 14 are mounted to or on the carrier 16 so that the sensors14 can be worn by an operator of the computer 12. One embodiment of thecarrier 16 is a wristband that places the sensors 14 against a person'swrist or forearm as illustrated in FIG. 1. This permits the sensors 14to easily monitor a person's blood pressure, pulse, body temperature, orskin conductivity. The carrier 16 may also be a headphone type carrier16 for placing the sensors against a person's ear and head.

The sensors 14 may also be attached to or mounted on a device that isheld by an operator so that the sensors 14 do not have to be worn by theoperator. For example, the sensors 14 may be incorporated into acomputer mouse so that the sensors 14 monitor an operator's physicalcondition while the operator is manipulating the mouse. For thisembodiment, the mouse would preferably include an orifice or a hole inwhich an operator could place one of his or her fingers. The sensor orsensors 14 would be mounted in the hole to measure the physicalconditions described above while the operator used the mouse.

The sensors 14 may also be incorporated into a computer joystick so thatthe sensors 14 could monitor a physical condition of an operator whileplaying a computer game or before starting the game. As with the mouse,the joystick would include an orifice or a hole for receiving anoperator's finger. The sensors may also be incorporated into otherdevices such as a table pad, a mouse pad, a desk, a chair, a floor mat,a hand print device, a foot print device, or even an automatic shouldermassager.

The computer interface 18 receives sensor signals from the sensors 14and delivers the signals to the computer 12. In preferred forms, thecomputer interface 18 is a universal serial bus (USB) or mini-USBinterface that couples with a conventional USB port on the computer 12.The computer interface 18 may also be a PS2 mouse type interface, anSCSI interface, an RS-232C interface, a parallel interface, a serialinterface, or even a wireless interface.

The computer program is operable for receiving the sensor signals fromthe human interface device 10 and for controlling certain operations ofthe computer 12 in response thereto. The program is stored on computerreadable memory accessible by the computer 12 such as the computer'shard drive and may be written in any computer language as a matter ofdesign choice.

As illustrated in FIG. 3, the computer program broadly includes a signalinterface component 20, a user interface component 22, a signal analysiscomponent 24, and a trend recording component 26. The signal interfacecomponent 20 receives the sensor signals from the human interface device10 and performs any necessary error checking functions. The userinterface component 22 provides a graphical user interface on themonitor of the computer 12 to allow an operator to control certainfunctions of the computer program as described in more detail below. Thesignal analysis component 24 analyzes the sensor signals received fromthe human interface device 10 and controls certain operations of thecomputer 12 or other computers as described in more detail below. Thetrend recording component 26 records the sensor signals along with otherinformation in a database that may be accessed for monitoring andcontrol purposes.

The flow diagram illustrated in FIG. 4 depicts several steps performedby the computer program for one application of the present invention.The computer program is initiated in step 400 when an operator of thecomputer 12 connects the computer interface 18 to the computer 12. Theoperator then puts on the wristband carrier 16 or other carrier so thatthe sensors 14 may begin to measure certain physical conditions of theoperator and transmit corresponding sensor signals to the computer 12.

The computer program preferably first monitors the physical conditionsof the operator before the operator has begun using the computer 12 toobtain resting sensor signals as depicted in step 402. The operator maythen view the resting sensor signals on the monitor of the computer 12and adjust the sensor signals to accommodate for irregularities asdepicted in step 404. For example, if the operator is currentlyexperiencing a high pulse rate due to recent exercise, the operator maymanually change the resting sensor signals accordingly.

The computer program next sets threshold levels corresponding to theresting sensor signals as depicted in step 406. The threshold levels aretypically some percentage of the resting sensor signals. For example,the threshold levels may be + or −30% of the measured resting sensorsignals from each of the sensors 14 for a low alarm and + or −50% of themeasured resting sensor signals from each of the sensors for a highalarm.

Once the computer program has received and stored the resting sensorsignals and set the threshold levels, the operator may begin to use thecomputer 12. For example, the operator may begin playing a computer gameor using an application program stored on the computer 12. While theoperator uses the computer 12, the human interface device 10 continuesto measure the physical conditions of the operator and to sendcorresponding sensor signals to the computer 12 as depicted in step 408.The computer program compares the current sensor signals to thethreshold levels as depicted in step 410 and displays both the currentsignals and the threshold levels on the computer 12 monitor as depictedin step 412.

In accordance with one important aspect of the present invention, thecomputer program may control certain aspects of the computer 12 based onthe received sensor signals and the comparison steps. For example, thecomputer program may determine that an operator who is playing acomputer game is not challenged based on the measurement of theoperator's physical conditions. The computer program may then increasethe speed or difficulty of the computer game so that the game is moreexciting to the operator.

Similarly, the computer program may determine that an operator who isusing a certain application program is experiencing high levels ofstress and/or fatigue. The computer program may then reduce the workloadof the operator and/or alert the operator or a supervisor that theoperator needs a break or assistance.

The computer program also preferably stores and records the sensorsignals along with other information in a database for later analysis asdepicted in step 414. For example, the computer program may store thesensor signals and the date and time each instance when an operator'scurrent sensor signals exceed the threshold levels. A supervisor orother person may then analyze the information in the database formonitoring purposes.

In another embodiment of the present invention, several computers 12 andhuman interface devices 10 may be connected to or through a hostcomputer 28 via a communications network 30 as depicted in FIG. 2. Thehuman interface devices 10 may also be connected to the host computervia mobile phones with Internet capabilities or even televisions withInternet capabilities. The communications network 30 may be anyconventional network such as the Internet, a local area network, a widearea network, or even a wireless network. The communications network mayalso link several personal computers 12 directly without use of the hostcomputer 28.

The host computer 28 is used to remotely monitor physical conditions ofoperators of the computers 12 or to provide information to the computers12. For example, in one application, the host computer 28 may beoperated by a supervisor to monitor the physical conditions of theoperators of the computers 12 to ensure that the operators are not toofatigued or stressed. The host computer may also be used to monitor thephysical conditions of pilots, drivers, or others that are beingmonitored by the operators of the computers 12.

In another application, the host computer 28 may be used to conductremote interviews. An interviewee wears the sensors 14 or places theirhands on the mouse or joystick and uses one of the computers 12 to videoconference with an interviewer operating the host computer 28. Thecomputer program running on the personal computers 12 provides sensorsignals to the host computer 28 corresponding to the monitored physicalconditions of the interviewees. This application provides valuableinformation to interviewers that cannot be obtained by conventionalmethods.

In another application, the host computer 28 may be used to remotelymonitor a person's physical conditions for health reasons. For example,people who are recovering from illnesses or accidents at home may bemonitored by a doctor or hospital operating the host computer 28.

In another application, the host computer 28 may be used for onlineremote marketing research type services. For example, selected consumersmay be instructed to wear the sensors 14 and then watch certaincommercials presented to them on the computers. The operator of the hostcomputer 28 may then collect and analyze sensor signals corresponding tothe physical conditions of the consumers to determine how they respondto certain types of commercials.

In another application, the host computer 28 may be used for remoteeducation or training purposes. For example, students may be instructedto wear or touch the sensors 14 while taking a class via one of thecomputers 12. A teacher may then monitor the physical conditions of thestudents to determine if a lecture is of the appropriate difficulty orspeed.

Although the invention has been described with reference to thepreferred embodiment illustrated in the attached drawing figures, it isnoted that equivalents may be employed and substitutions made hereinwithout departing from the scope of the invention as recited in theclaims.

1. A method of generating and using information regarding a physicalcondition of an operator of a computer running a computer program, themethod comprising the steps of: (a) measuring with a sensor the physicalcondition of the operator when the operator is in a rest state, andgenerating a corresponding rest state sensor signal; (b) measuring withthe sensor the operator's physical condition that is a direct responseto the operator's interaction with the computer program, and generatinga corresponding active state sensor signal; (c) comparing the rest statesensor signal to the active state sensor signal; and (d) changing anaspect of the computer program when the active state signal differs fromthe rest state signal by a predetermined amount.
 2. The method as setforth in claim 1, wherein the physical condition is selected from thegroup consisting of: blood pressure, pulse rate, body temperature,static capacity, skin conductivity, brain wave activity, NO₂ blooddensity, NO₃ blood density, static electricity, and bloodcharacteristics.
 3. The method as set forth in claim 1, wherein thesensor is incorporated into a device selected from the group consistingof: wristband, headphone, mouse, joystick, desk, chair, and floor mat.4. The method as set forth in claim 1, wherein the computer program is acomputer game, and the aspect of the computer game relates to a level ofdifficulty of play, such that, in step (d), the level of difficulty ofplay is changed when the active state signal differs from the rest statesignal by the predetermined amount.
 5. The method as set forth in claim1, wherein the computer program is a computer game, and the aspect ofthe computer game relates to a speed of play, such that, in step (d),the speed of play is changed when the active state signal differs from,the rest state signal by the predetermined amount.
 6. The method as setforth in claim 1, further including the step of (e) communicating awarning or a suggestion to the operator based on the physical conditionof the operator as evidenced by an amount of difference between the reststate sensor signal and the active state sensor signal.
 7. The method asset forth in claim 1, further including the step of (e) communicating awarning or a suggestion to a supervisor of the operator based on thephysical condition of the operator as evidenced by an amount ofdifference between the rest state sensor signal and the active statesensor signal.
 8. The method as set forth in claim 1, further includingthe step of (e) storing the rest state sensor signal and the activestate sensor signal and a date and a time for use in analyzing workingconditions based on a change in the physical condition of the operatorduring operation of the computer program as evidenced by an amount ofdifference between the rest state sensor signal and the active statesensor signal.
 9. The method as set forth in claim 1, further includingthe step of (e) storing the rest state sensor signal and the activestate sensor signal and a date and a time for use in analyzing workingconditions based on a change in the physical condition of the operatorduring a specific period of time as evidenced by an amount of differencebetween the rest state sensor signal and the active state sensor signal.10. The method as set forth in claim 1, further including the initialstep of connecting the sensor with a remote host computer using acommunication link provided by a wireless telephone, wherein steps (c)and (d) are performed by the remote host computer.
 11. The method as setforth in claim 1, further including the initial step of connecting thesensor with a remote host computer using a communication link providedby a television adapted to access a network, wherein steps (c) and (d)are performed by the remote host computer.
 12. A method of generatingand using information regarding a physical condition of an operator of acomputer presenting an advertisement, the method comprising the stepsof: (a) measuring with a sensor the physical condition of the operatorwhen the operator is in a rest state, and generating a correspondingrest state sensor signal; (b) measuring with the sensor the physicalcondition of the operator when the operator is exposed to theadvertisement presented on the computer, and generating a correspondingactive state sensor signal; (c) comparing the rest state sensor signalto the active state sensor signal; and (d) analyzing the advertisementas a function of an effect of the advertisement on the operator asevidenced by a difference between the rest state sensor signal and theactive state sensor signal.
 13. A method of generating and usinginformation regarding a physical condition of an operator of a computer,wherein the operator is engaged in an interview via the computer, themethod comprising the steps of: (a) measuring with a sensor the physicalcondition of the operator when the operator is in a rest state, andgenerating a corresponding rest state sensor signal; (b) measuring withthe sensor the physical condition of the operator when the operator isengaged in the interview via the computer, and generating acorresponding active state sensor signal; (c) comparing the rest statesensor signal to the active state sensor signal; and (d) analyzing aneffect of the interview on the operator as evidenced by a differencebetween the rest state sensor signal and the active state sensor signal.14. A method of generating and using information regarding a physicalcondition of a subject engaged in an activity, the method comprising thesteps of: (a) linking a sensor to a host computer; (b) measuring withthe sensor the physical condition of the subject when the subject is ina rest state, and generating a corresponding rest state sensor signal,and transmitting the rest state sensor signal to the host computer; (c)measuring with the sensor the physical condition of the subject when thesubject is engaged in an activity, and generating a corresponding activestate sensor signal, and transmitting the active state sensor signal tothe host computer; (d) comparing, at the host computer, the rest statesensor signal to the active state sensor signal; and (e) communicatingan effect of the activity on the subject as a function of a differencebetween the rest state sensor signal and the active state sensor signal.15. The method as set forth in claim 14, wherein the subject is astudent and the activity involves an educational activity.
 16. A methodof permitting a first person and a second person to interact, the methodcomprising the steps of: (a) measuring with a sensor worn by the firstperson, wherein the first person is operating a first device, a physicalcondition of the first person and generating a sensor signal; (b)transmitting the sensor signal to the second person, wherein the secondperson is operating a second device, via a communications network; and(c) displaying the sensor signal on the second device so that the secondperson can interact with the first person based upon the sensor signal.17. A method of generating and using information regarding a physicalcondition of an operator of a vehicle, the method comprising the stepsof: (a) measuring with a sensor the physical condition of the operatorbefore operating the vehicle, and generating a corresponding rest statesensor signal; (b) measuring with the sensor the physical condition ofthe operator while the operator is operating the vehicle, and generatinga corresponding active state sensor signal; (c) comparing the activestate sensor signal to the rest state sensor signal so as to monitor theoperator's physical condition while operating the vehicle; and (d) ifthe active state sensor signal is outside a predetermined acceptablerange for the operator while operating the vehicle, alerting a thirdparty that is not the operator engaged in the activity.